Papers with language-independent representations
MauBERT: Universal Phonetic Inductive Biases for Few-Shot Acoustic Units Discovery (2026.acl-long)
Copied to clipboard
| Challenge: | MauBERT models learn from multilingual data to predict articulatory features or phones, resulting in language-independent phonetic representations. |
| Approach: | They introduce a multilingual extension of HuBERT that leverages articulatory features for robust cross-lingual phonetic representation learning. |
| Outcome: | The proposed model can predict phonetic features in 55 languages with minimal fine-tuning (10 hours of speech) it is more context-invariant than state-of-the-art models and adapts to unseen languages and casual speech with minimal self-supervised fine- tuning (10 hours) |
Language-Independent Representations Improve Zero-Shot Summarization (2024.naacl-short)
Copied to clipboard
| Challenge: | Pretrained models can be fine tuned on downstream generation tasks, but they can fail in zero-shot conditions. |
| Approach: | They propose query-key finetuning to decouple task-specific knowledge from pretrained models . they propose a variant that more directly enforces language-agnostic representations . |
| Outcome: | The proposed model decouples task-specific knowledge from pretrained language generation abilities. |
Low-Resource Sequence Labeling via Unsupervised Multilingual Contextualized Representations (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual sequence labeling require bilingual resources and require linguistic knowledge. |
| Approach: | They propose a multilingual language model with deep semantic Alignment to generate language-independent representations for cross-lingual sequence labeling. |
| Outcome: | The proposed model achieves state-of-the-art NER and POS performance across European languages and on distant language pairs such as English and Chinese. |
Improving Zero-Shot Translation by Disentangling Positional Information (2021.acl-long)
Copied to clipboard
| Challenge: | Multilingual neural machine translation has shown the capability of directly translating between language pairs unseen in training, i.e. zero-shot translation. |
| Approach: | They propose to remove residual connections in an encoder layer to reduce the difficulty of generalizing to new translation directions. |
| Outcome: | The proposed model outperforms pivot-based translation in terms of quality and ease of integration of new languages. |
Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks (2022.naacl-main)
Copied to clipboard
| Challenge: | Quantifiers are pervasive in NLU benchmarks and their occurrence at test time is associated with performance drops. |
| Approach: | They propose a generalized quantifier NLI task to quantify their contribution to the errors of NLU models. |
| Outcome: | The proposed model is based on a generalized quantifier theory and is compared with pre-trained models. |